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ABSTRACT 


This  paper  introduces  a  technique  of  smoothing  high  frequency  random 
fluctuations  from  noisy  data.  An  optimum  lowpass  filter  is  developed  by  using 
the  circular  serial  correlation  statistic  to  minimize  the  random  noise  content 
in  serial  data.  The  procedure  is  applicable  for  statistically  mild  data.  All 
the  basic  aspects  of  the  new  filter  technique  are  discussed  and  are  applied  to 
both  simulated  noisy  data  and  actual  measured  data  of  unknown  noise 
characteristics.  Not  only  is  the  procedure  applicable  to  noise  reduction,  but 
it  also  appears  to  provide  a  means  to  reconstruct  undersampled  data. 

A 

Introduction 

There  are  several  methods  of  estimating  the  intelligence,  or  signal, 
components  of  noisy  data  [1,2].  Most  of  these  methods  involve  a  least-square 
error  approach  either  directly  or  indirectly  [3].  In  the  conventional  least 
square  (LS)  approaches,  an  error  signal  is  used  to  determine  a  set  of  system 
parameters,  usually  the  constant  coefficients  in  a  difference  equation.  In 
some  of  these  LS  approaches,  either  the  order  of  the  data  is  required  or  its 
estimate,  since  the  number  of  coefficients  is  often  assumed  to  be  known. 
Whenever  the  system  order  is  unbounded,  as  occurs  in  the  singularity  expansion 


method,  it  is  expected  that  the  conventional  LS  procedures  will  have 
difficulties  [4]. 

It  is  well  known  that  a  truncated  Fourier  series  provides  an  LS  fit  to 
any  function  that  is  bounded  and  piecewise  differentiable  in  a  finite  interval 


[5].  A  procedure  is  presented  for  obtaining  the  optimum  truncation  by 
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examining  the  randomness  of  the  time  series  formed  by  the  differences  between 
the  original  data  and  the  truncated  Fourier  series.  The  relative  randomness 
is  determined  by  using  the  circular  serial  correlation  coefficient  [6].  Since 
the  truncated  Fourier  series  is  equivalent  to  the  result  of  implementing  a  low 
pass  filter,  the  foregoing  procedure  yields  the  optimum  lowpass  filter  in 
reducing  the  random  noise  contribution  to  the  data,  provided  the  data  are 
statistically  mild.  9 

In  other  methods,  for  example,  adaptive  [7-9]  and  Kalman  [10,11] 
filtering,  a  prior  knowledge  of  noise  components  mean,  variance  and 
correlation  properties  are  needed.  In  adaptive  filtering  it  is  well  known 
that  the  best  LS  estimate  of  the  primary  signal  is  produced  by  adjusting 
filter  parameters,  i.e.,  minimizing  the  LS  error  power.  At  the  expense  of  not 
necessarily  being  the  best  LS  representative,  our  adaptive  LS  estimate  is 
produced  by  maximizing  the  randomness  of  the  error  through  changing  the  set  of 
Fourier  coefficients.  Also,  it  is  possible  to  recast  the  Fourier  series  in 
terms  of  state  variables  for  Kalman  filtering  analysis.  The  solution  to  this 
problem,  however,  additionally  assumes  a  known  initial  state  and  involves  a 
set  of  matrix  inversions.  Our  nonparametric,  orthogonal  polynomial  approach 
is  a  simple  straight  forward  method,  only  requiring  a  statistical  test  of 
randomness  of  the  error  signal  as  opposed  to  using  ad  hoc  rules  of  thumb  in 
making  this  type  test.  The  test  statistic  only  requires  2N  additions  and 
multiplications  calculations.  The  statistic  used  in  making  the  randomness 
test  is  similar  to  the  non-normal ized  version  of  the  correlation  statistic 
recently  used  by  Pakula  and  Kay  [12]  in  their  radar  detection  work.  It  is  a 
circularly  defined  single  lag  autocovariance  statistic  well  known  in  the 


mathematical  statistics  area[13,14]  and  the  spectral  analysis  detection 
area[15,16,17].  However,  in  our  application  of  the  “circular  correlation" 
statistic,  a  bow-tie  correction[17 ,18]  for  spectral  bias  error  and  windowing, 
is  not  required  since  the  correlation  function  is  calculated  directly  in  the 
"time"  domain.  The  method  presented  here  is  a  simple  utilization  of  a  novel 
randomness  test  to  construct  a  low-pass  filter. 

The  flow  diagram  for  the  circular  serial  autocorrelation  filter  (serial 
correlator  filter)  is  shown  in  Fig.  1.  It  shows  the  principle  processing  used 
to  obtain  an  estimate  signal  from  the  noisy  signal.  The  basic  aspects  of  this 
new  technique  for  filtering  noisy  data  are  considered  in  the  following 
discussion.  In  the  first  section,  fundamental  characteristics  of  a  Fourier 
integrable  noisy  signal  are  presented.  The  next  section  includes  a  discussion 
of  the  estimation  of  the  desired  signal.  Also  presented  in  this  section  are 
the  statistical  bases  of  the  filter  which  allow  the  extraction  of  an  estimate 
of  the  desired  signal.  Salient  features  of  the  correlation  coefficient  and 
what  is  meant  by  statistically  mild  data  are  also  discussed.  The  third 
section  provides  a  brief  treatment  of  the  noise  estimation  problem.  In  this 
section,  rough  estimates  of  signal-to-noise  ratios  related  to  the  noisy  signal 
nd  to  the  estimated  desired  signal  are  presented.  Lastly,  several  examples 
of  the  actual  use  of  the  filter  are  given. 


Noisy  Signals 

A  real  noisy  signal  is  assumed  to  have  two  summable  components:  a 
deterministic  component  and  a  non-deterministic  component.  The  deterministic 
component  is  called  intelligence  and  the  non-deterministic  component  is  called 
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Figure  1:  Basic  Diagram  of  Correlation  Filter 
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original  noise.  If  fn,  f,  and  n  respectively  denote  the  noisy  signal, 
intelligence  signal,  and  original  noise  signal,  then  fn  obeys  the  relation 

fn(t)  -  f(fc)  +  «(t)  (1) 

where  t  is  the  independent  variable.  n(t)  for  each  t  is  a  value  assumable  by 
a  stationary  random  noise  variable.  f(t)  is  assumed  to  be  a  response  of  a 
deterministic  and  stationary  system.  In  discrete  cases  for  N  even  samples  of 
sampling  period  T,  where  subscript  k  denotes  the  discrete  independent 
variable,  the  noise  contaminated  signal  of  equation  (1)  is  represented  by 

^nk  *  ^k  +  nk  k-0,1 . N-l.  (2) 

The  intelligence  signal-to-original  noise  ratio  in  dB,  defined  as  the 
ratio  of  the  rms  value  of  fk  to  the  rms  value  of  n^,  is  given  by  the  usually 
unknown  quantity* 

rms  ( f .  } 

SNR1  •  20,°9  rmsTn^T  •  <3) 

To  determine  an  estimate**  of  SNRj  and  to  eliminate  portions  of  nk,  it  is 
sufficient  to  determine  the  estimates  of  the  intelligence  signal  f^  (the 

*The  rms  value  of  a  discrete  signal  is  approximately  its  standard  deviation, 
if  its  average  value  is  negligible. 

**An  estimated  quantity  is  denoted  by  a  small  circumflex  mark  over  the  top  of 
the  basic  symbol.  If  the  value  of  the  subscript  is  not  indicated,  the 
subscripted  quantity  represents  the  entire  waveform. 


symbol  f^)  and  (the  symbol  n^).  These  estimates  are  discussed  in  the 
following  sections. 


Estimation  of  Intelligence 

In  some  conventional  LS  approaches  either  the  order  of  the  data  is 
required  or  its  estimate.  The  truncated  Fourier  series  estimate  of  the  noise 
contaminated  signal,  however,  eliminates  the  necessity  of  prior  knowledge  of 
order.  On  the  other  hand,  the  Fourier  series  (f  ^  representation  retains 
portions  of  random  noise  components  which  share  the  same  frequencies  as  the 
harmonics  in  the  representation.  This  distorts  the  series  representation. 
Since  in  finite  applications  the  Fourier  series  must  truncate  at  some  harmonic 
(L) ,  the  difference  signal  f^  minus  f  ^  may  contain  deterministic  as  well  as 
non-deterministic  components.  This  causes  additional  concerns.'  These  two 
problems  are  related  since  the  Fourier  series  is  the  noisy  signal  minus  the 
difference.  In  the  special  case  of  a  zero  mean  nk,  it  is  easily  shown  that 
the  estimated  signal  (f^)  is  exact  for  a  diagonal  autocovariance  difference 
matrix;  thus,  the  latter  problem  is  eliminated  because  in  this  case  the 
difference  signal  is  purely  random. 

As  shown  in  Appendix  A  a  stationary  random  error  (the  difference  between 
intelligence  and  its  estimate)  implies  an  intelligence  estimate  whose 
deterministic  part  is  the  ideal  intelligence  f^.  It  also  is  easily  shown  that 
when  intelligence  is  estimated  exactly,  the  error  is  random  regardless  of  the 
noise  mean.  We  assume  when  the  difference  signal  is  significantly  random 
that  f  k  is  a  reasonable  estimate  of  f^.  That  is,  the  estimate  of  the 
intelligence  is  related  to  the  Fourier  series  estimate  for  significantly 
random  error  by  the  approximation 


where  L  signifies  the  dependency  of  the  estimate  of  f^  on  the  number  of 
harmonics  retained  in  the  Fourier  series.  By  definition,  therefore,  we  take 
as  an  estimate  of  the  intelligence,  the  following  truncated  Fourier  series, 
where  &iQ  -  2it/[(N-1)T]  is  the  fundamental  frequency: 

L 

fk  L  “  Ao  +  E  [Amcos(m(D0kT)  +  Bmsin(ma>0kT)].  (5) 

1  m=*l 

The  above  Fourier  coefficients  are  given  by  either  a  fast  Fourier  transform  of 
a  properly  sampled*  fn|,,  or  by  some  archaic  evaluation  of  the  following 
integrals  over  a  fundamental  period: 

(6) 

Am  "  T  j  fn(t)  cos(m<o0t)dt  m-1 . L  (7) 

0  0 

Bm  "  T  f  °  fn(t)  sin(mw0tJdt  m-1, — ,L  .  (8) 

1  o  o 
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ytj  * 


*  If  data  are  not  properly  sampled  aliasing  causes  the  OFT  method  to  give  poor 
results  for  the  coefficients. 


fn(t)  is  obtained  from  the  sample  f^  via  interpolation.  Both  linear  and 
quadratic  interpolation  have  been  considered  with  comparable  results  obtained 
in  all  cases. 


In  attempting  to  generate  a  non-deterministic  curve-fit  error  the  order, 
L,  of  the  harmonic  expansion  is  incremented  to  its  maximum  value  determined  by 
the  Nyquist  sampling  interval.  For  each  L  an  error  signal  is  defined  by  the 


difference  between  the  noisy  signal  and  the  intelligence  estimate: 


eLk  “  fnk  ’  fk  L 


This  signal  has  certain  characteristics.  By  definition,  it  tends  not  to  be 
slowly  oscillatory.  It  is  not  the  difference  between  and  the  unknown  fk. 
Its  rms  value,  a  figure  of  merit  denoted  by  EL, 


El  -  rms  (eLk)  ,  (10) 

is  slowly  varying  over  a  range  of  L  values.  Thus,  its  minimum  power  as  a 
function  of  L  is  not  necessarily  located  at  a  finite  turning  point.  Finally, 
its  randomness  is  a  function  of  L.  A  method  of  deciding  when  this  signal  is 
random  follows. 

Another  figure  of  merit  of  the  error  signal  is  its  circular  serial 
correlation  coefficient.  An  exact  distribution  of  the  (circular)  serial 
correlation  coefficient  for  random  time  series  is  known  [6,13,14].  The  Wald 
and  Wolfowitz  "correlation"  statistic  [6]  for  N  sample  values  of  detrended  e>  l. 


(11) 


N-l 

RL  "  eL,OeL,N-l  +  eL,k-leL,k 

This  simple  single  lag  correlation  measure  is  a  function  of  L.  R|_  has  a  mean 
and  variance  for  random  signals  given  respectively  by 


yR  -  (sx2  -  S2)/(N-1) 


2  S2  -  S4  sj  -  4S2S2  +  4SjS3  +  S2  -  2S4  (S21  -  S2)2 

aRL  ’  N-l  +  (N-l)  ( N— 2)  (n_1}2 


(12) 


(13) 


where  is  the  i^  moment  of  the  detrended  e^  about  the  origin.  Other 
statistical  characteristics  of  the  error  are  also  central  to  this  work,  and  we 
need  more  than  the  above  correlation  moments  to  structure  a  randomness  test. 

At  this  point,  statistically  mild  data  is  defined  and  relevant  theorems 
cited.  Statistically  mild  data  are  data  1)  whose  moments  are  bounded  for  all 
N,  and  2)  whose  variance  is  greater  than  zero  for  N-*».  (N  is  greater  than  75 
for  asymptotic  properties  of  the  data.)  In  applications  of  our  method,  if 
data  values  are  bounded  and  no  single  value  is  exactly  equal  to  the  mean  of 
the  data,  then  the  data  are  statistically  mild  as  provided  in  Theorems  1  and  2 
of  Appendix  B.  Thus,  each  finite  error  value  should  be  compared  to  the  error 
mean  to  test  for  mildness  before  the  above  correlation  statistics  are  applied. 

The  distribution  of  R|_  is  asymptotically  normal  for  statistically  mild 

data.  For  "large"  N  a  test  statistic  which  is  a  normalization  of  RL> 


can  be  used  to  structure  a  statistical  test  of  the  hypothesis  that  e|_k  is 
random  [14].  The  test  for  randomness  proceeds  as  follows.  The  probability 
that  |ZJ  has  a  value  less  than  or  equal  to  a  positive  Z0  is 

P  (|ZL|S  ZQ)  -  i!  Jq  0  exp  [-u2  /  2]  du.  (15) 

The  null  hypothesis  (HQ)  is  that  the  error  signal  is  random,  i.e.,  ZL«0;  and 
the  single  alternate  hypothesis  (H^)  is  that  eLk  is  not  random  (eLk  is 
oscillatory  or  has  a  trend),  i.e.,  ZL  *0.  Utilizing  the  .05  significance 
level,  the  null  hypothesis  is  rejected  if  |ZL|  >  1.96.  Conversely,  the  null 
hypothesis  is  accepted  if  [Zj  <  1.96.  Consequently,  in  the  search  for  an 
optimum  L,  the  test  statistic  ZL  is  calculated  at  each  step  and  the  value  of  L 

for  which  |ZJ  is  smallest  is  the  optimum  value  provided  |ZL|  <  1.96. 

However,  it  is  important  to  note  that  there  is  not  a  100%  certainty  the  error 
is  random  even  if  a  ZL  is  found  which  is  exactly  zero.  In  this  case,  the 

outcome  of  the  test  for  randomness  would  be  exactly  the  same  as  it  would  be  if 

the  Zl  were  1.95.  Therefore,  we  avoid  the  proposition  that  a  Z^_  -  0  implies 
statistically  independent  error  samples;  and  more  importantly,  we  use  the 
confidence  of  the  truth  of  the  alternate  hypothesis  in  deciding  the  randomness 
of  the  error  samples. 

There  obviously  are  other  approaches  to  determining  an  optimum  L.  The 
rms  value  of  the  error  signal  as  described  through  equation  (10)  may  be  useful 
in  this  purpose.  Its  value  can  be  minimized,  as  is  done  in  many  other 


filtering  methods.  In  cases  of  f^'s  for  which  either  the  noise  components 
are  known  to  be  small  or  Z|_  is  not  minimizable  in  [l,N/2),  setting  a  limit  on 
the  curve-fit  error  can  be  an  important  requirement  for  determining  the 
estimate  waveform.  Then  iterations  for  selection  of  an  optimal  L  can  be 
stopped  when  the  error  is  either  less  than  the  set  limit  or  locally  minimum. 

If  large  noise  components  are  known  to  be  present,  however,  the  former 
statistical  approach  should  be  used.  As  shown  in  the  first  example, 
minimizing  the  error  may  not  improve  the  errors  randomness  and  may  not  be 
useful.  In  cases  where  the  Fourier  series  is  not  the  desired  method  to  use  in 
generating  the  error  signal,  other  eigenfunction  expansions  can  be  used. 

Generally,  f^  can  be  represented  by  a  weighted  sum  of  orthonormal 
functions  (<J>m's)  orthogonal  over  some  determined  interval,  Ta  £  t  S  T^,  i.e., 

\|L  •  Jo  W">  (I6) 


where  the  constants  are  given  by  discrete  versions  of  the  integrations 


cm  -  [  Jb  fn(t)+m(t)dt]  /  [  JTb  *m2(t)dt] 

61 


(17) 


When  a  more  general  set  of  expansion  functions  is  used,  the  error  signal  and 
error  figure  of  merits  can  be  defined  as  in  the  Fourier  LS  approach.  All  of 
the  preceding  ideas  can  be  restated  in  terms  of  the  new  expansion.  In  any 
signal  estimation  method  used,  however,  the  resulting  random  error  signal  is 


an  estimate  of  the  random  noise  content  of  f, 


iik* 


-11- 


Estimation  of  Noise 


An  estimate  of  nk  is  taken  to  be  the  error  signal  when  the  number  of 
terms  in  the  series  is  optimum.  It  follows  from  the  form  of  equation  3  that 
an  estimate  of  the  relative  random  noise  content  of  fnk  is  simply  20  times  the 
common  log  of  the  ratio  of  the  rms  value  of  the  estimates  of  fk  to  the  rms 
value  of  the  estimate  of  nk: 


SNR, 


201og 


rms(fk  L.) 


rms(eL,k) 


(18) 


where  L1  is  the  optimum  value  of  L.  Other  estimates  of  SNR^  are  possible  if 
various  combinations  of  error  signals  and  estimates  of  fk  are  used  in  equation 
18 . 

The  noise  content  of  the  filtered  signal  also  can  be  estimated.  The  SNR 
given  by  the  following  formula  is  an  estimate  of  the  SNR  of  f  k: 


rms(fk) 

SNR3  -  201og  - * - 

rms<fnk  -  fk  -  eL'k> 


(19) 


where  fk  is  an  estimate  of  fk>  SNR^  is  a  measure  of  the  remaining  noise  in 
the  estimate  of  the  intelligence  signal.  Equation  19  defines  a  good  estimate 
when  fk  and  fk  are  approximately  equal.  For  large  differences  between  fk  and 
fk,  equation  19  should  not  be  used.  The  utility  of  the  former  equation  18  is 
indicated  in  the  following  discussion  of  application  examples. 
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Application  Examples 

Plots  of  original  noisy  signal  results  are  given  in  Figs.  2-6.  Figure  2 
represents  simulated  data  which  has  a  20  dB  SNR  and  its  continuous  parent 
intelligence  obeys  the  following  equation: 

f(t)  -  3e”^sin  lOt  +  e”^cos  5t.  (20) 

Figures  3  and  4  are  the  error  figures  and  Z-statistic  plots  respectively  for 
the  simulated  data.  Figure  3  shows  that  the  error  does  not  change  much 
between  15  and  40  harmonic  representations .  Figure  4  shows  that  the  optimum 
value  for  retained  harmonics  is  17  for  the  data. 

The  noise  estimate  according  to  equation  (18)  is  19.9  dB  for  straight 
line  detrended  error  for  the  simulated  data,  to  which  20  dB  SNR  pseudo  random 
noise  was  added.  Figure  5  represents  an  undersampled  aircraft  transient 
voltage  measurement  and  Fig.  6,  an  aircraft  current  measurement.  The  nature 
of  the  contaminating  noise  is  not  known  in  these  data.  The  estimate  SNR  for 
the  voltage  measurements  of  Fig.  5  is  18.9  dB. 

The  filtered  versions  of  the  data  sets  are  given  in  Figs.  7-10.  Aircraft 
current  measurements  plots  are  not  overlayed  as  in  the  other  data  plots 
because  of  the  difficulties  in  distinguishing  distinct  graphs.  Figure  9  is 
merely  an  FFT  reproduction  of  Fig.  6;  Figure  10  is  the  filtered  aircraft 
current  data.  Figures  7  and  8  show  the  performance  of  the  filter  in  more 
comparable  fashions  and  correspond  to  the  data  of  Figs.  2  and  5, 
respectively.  Figure  7  indicates  the  smoothing  effects  of  the  filter  and  its 
inability  to  remove  all  noise.  Figure  8  shows  that  the  filter  "reconstructs" 


Figure  3:  RMS  Value  of  Error  Signal 
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an  intelligence  estimate  and,  only  in  that  sense,  compensates  for 
undersampling.  Finally,  Fourier  coefficients  for  the  data  of  Figs.  2  and  5 
are  respectively  given  in  Tables  1  and  2. 

For  the  data  that  are  presented,  linear  interpolation  is  used  to  compute 
the  Fourier  coeff icients.  If  the  discrete  Fourier  series  coefficients  are 
used,  the  truncation  of  the  series  represents  the  result  of  applying  an  ideal 
zero-phase  lowpass  filter.  With  linear,  quadratic,  etc.  interpolation  used  in 
equations  (6)  -  (8),  there  will  be  some  resulting  phase  and  amplitude 
distortion,  albeit  very  small.  Each  computational  scheme  seems  to  have  its 
own  advantages  and  disadvantages.  However,  the  associated  differences  in  the 
filtered  signal  are  generally  within  the  residual  noise. 

Conclusions 

A  simple  technique  for  reducing  random  noise  components  in  serial  data 
has  been  presented.  Basic  theory  required  for  determining  the  estimate  signal 
has  been  given  along  with  information  on  the  curve-fit  error  single  lag, 
autocorrelation  coefficient.  Statistically  determined  cutoff  in  harmonic 
content  of  signals  has  been  used  in  truncating  (Fourier)  least  squares 
curve-fits.  Verification  of  the  procedure  was  provided  by  considering 
examples  of  results  when  the  filter  is  applied  to  simulated  data  sets  as  well 
as  actual  measured  data  where  the  noise  characteristics  are  not  known.  These, 
together  with  the  actual  estimates  of  noise  content,  indicate  that  serial 
correlation  filtering  is  a  viable  procedure  for  smoothing  random  data. 


TABLE  1.  FOURIER  COEFFICIENTS  OF  SIMULATED  DATA 
(T0  -  Fundamental  Period) 


AO 


WO 


T 


o 


.119067+000 


.249333+001 


200000-001 


HARMONIC 


B 


1 

.240617+000 

-.336733-001 

2 

.554411+000 

.124682+000 

3 

.215013+000 

.319436+000 

4 

.804515-001 

.332959+000 

5 

-.317570-001 

.257701+000 

6 

-.623676-001 

.158842+000 

7 

-.568762-001 

.105023+000 

8 

-.384506-001 

.889833-001 

9 

-.460509-001 

.560041-001 

10 

-.225412-001 

.634750-001 

11 

-.299089-001 

.469672-001 

12 

-.270673-001 

.400109-001 

13 

-.187330-001 

.247502-001 

14 

-.111443-001 

.120933-001 

15 

-.225465-001 

.109459-001 

16 

.244487-002 

.304705-001 

17 

-.126514-001 

.125419-001 

TABLE  2.  FOURIER  COEFFICIENTS  OF  AIRCRAFT  VOLTAGE  MEASUREMENTS 
(TQ  -  Fundamental  Period) 


AO 

WO 

To 

-.504385-002 

.126652+009 

.390630-009 

HARMONIC 

A 

B 

1 

-.339958-002 

-.116889-001 

2 

-.299620-003 

.102555-002 

3 

-.982244-003 

-.354796-002 

4 

.123119-002 

-.149075-002 

5 

.301493-003 

.211356-002 

6 

-.279008-002 

-.310397-002 

7 

.232368-002 

.155552-002 

8 

.177139-003 

-.519210-003 

9 

.598966-003 

-.419900-002 

10 

.491952-002 

.410862-002 

11 

.155511-002 

.186848-002 

12 

.332603-003 

.425356-003 

13 

.148213-002 

.100724-002 

14 

.728591-003 

.892899-003 

15 

.324246-003 

.503268-003 

16 

.125321-002 

.555634-003 

17 

.870975-004 

.378663-003 

18 

.422560-003 

.300633-003 

19 

.120060-002 

.810492-003 

20 

-.246067-003 

.568349-003 

21 

.138480-003 

.162885-003 

22 

.801859-003 

.678513-003 

23 

-.341879-003 

.615869-003 

24 

.129803-003 

.288792-003 

25 

.799167-003 

.208817-003 
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APPENDICES 


I 


Appendix  A 


Let  1)  k  be  the  discrete  variable  where  k  is  an  element  of  {0,1,. . . ,N-1} 
for  an  integer  N;  2)  ek  and  bk  be  stationary  random  noises  with  arbitrary 
moments;  3)  ak  and  fk  be  deterministic  signals;  4)  be  the  sum  of  ak  and 
-bk;  and  5)  f^  -  f^  -  ek  for  the  signal  fk.  It  follows  that  a^  -  fj. 

Proof:  From  conditions  5)  and  4) 


ek  ■  fk  '  ak  +  bk 


k-0,...,N-l  . 


( A—  1 ) 


& 


) 

vi.rJ 

•ft 

m 


Then  the  product  e^  is  given  by 


Vk  ■  Vk*  fiV  fiV  atV  Vr  a*V  biV  akbi*  bibk- 


(A-2) 


where  1  and  k  are  arbitrary  elements  of  {0,1... N-l}.  The  expected 
value  of  the  product  is 


£[etej  -  c  «k  -  (fj-  (fk-  ak)  +  [(f a,)  ♦  (fk-  a,,)]  e[b,]  (A-3) 


i  uiJ 


'k  k; 


8 

where  condition  2)  has  been  used,  c  is  a  constant  and  <5^  is  the  Kronecker 


delta. 


In  other  words,  if  (fj  -  a^)  +  (fk  -  ak)  is  not  zero  and  l  ■»  k, 


$ 


_  ,  -<V  a*>  (V  V 

Cbi]  “  (fr  a ~)  +  {fv-  aj 


'k  ak- 


{ A— 4 ) 


<ft  -  V  *  <fk  -  ak>  •  °- 


(A-5) 


From  equation  A-3  for  ink,  since  l  is  not  always  equal  to  k. 


f 4  -  a t  i  -  0.....N-1.  //  (A-5) 

Thus,  this  proof  shows  that  the  deterministic  part  of  the  estimate  ffc  is 
unique  under  the  stated  conditions  1)  through  5). 


Appendix  B 


Theorem  1  Let  N  and  r  be  positive  integers  and  ai  (i-l,...,N)  be 

real  numbers.  If  y  is  a  bounded  non-negative  real 
number  such  that  |a^|  £  p  (i-l,...,N),  then  there  is  a 
sequence  of  real  numbers  {Aj}  for  which 


for  all  N. 

Lemma  If  C  and  0  are  non-negative  real  numbers  such 
that  C  £  D,  then  CrS  Dr  (r-1,2, . . . ) . 

Proof  of  Lemma:  For  r-1  C  i  D,  by  hypothesis.  For  m  an 
integer  less  than  or  equal  to  r,  if 
Cm~l$  Dm_1  then  DCm"}s  Dm.  However, 
the  inductive  process  allows  the  case 
cm-(m-I)^  Q  which  imp!  ies  Cm  S  DC111'1. 
Thus,  Cm  S  Dm.  // 

Proof  of  Theorem  I:  Suppose  A^  -  pr.  By  hypothesis,  2  0  (j-l,...,r). 

The  hypothesis  and  the  above  lemma  imply  that 

N  r  r 
[  a,  i  Np  . 
i-1  1 

The  triangle  of  inequality  rule  and  the  rule  of 
exponentiation  imply  that 
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t 

I 

r-»  - 


a 


i 

l 


a 

•A 


r 


i 


•i 

•w 


vJ 

* , 


i 

$ 


*  I  a  <  h 


N,r-1, 2, . . . .  // 


Theorem  2 


Let  N  be  a  positive  integer,  ai  be  real  numbers 

1  N 

(i-l,...,N)  and  I  be  equal  to  ^  l  a^ .  If  a^  .*  ! 
(i-l,...,N)  and  0  is  a  real  number  equal  to 


min[min(a  ,-ra?  ],  then  there  is  a  real  number 
v  N  i  1 


Y  (y  >  0)  such  that 


lim  1  ?  fa  -?12  >  y 

N—  fl  (ai  a)  *  Y> 

T  »  i 


Proof  of  Theorem  2:  Suppose  y  ■  8.  By  hypothesis,  0  >  0.  Also, 


W  l  (ai-J)2  a  H  N8  -  B 


therefore, 


!»•  R  Tj  *  *:  " 


DT  C 


